A neural network is a network of many very simple processors, the processors also being known as neurons, nodes, units, or perceptrons. The processors are connected by unidirectional communication channels (connections), that carry numeric data. The processors operate only on their local data and on the inputs they receive by way of the connections. Numerical values called weights express the strength of each connection. The weights may be modified to increase or decrease the strength of the connections.
While there are many different types of neural networks, feed-forward neural networks are the most common. Feed-forward neural networks can generally be implemented as functions y(f,w) of a vector “f” of inputs and a weight or parameter vector “w”. Adjustment of vector w is referred to as training. Thus, instead of being pre-programmed to carry out a specific task, feed-forward neural networks have a “training” rule whereby the internal weights of each connection are adjusted according to data patterns introduced to each weight. While numerous different training methods are currently available, one of the most commonly used methods is error back-propagation.
Training of the internal weights or gains is difficult, especially when the neural network is implemented in hardware such as an analog computation device or chip. Many schemes have been developed to aid the insertion of the weights or gains into the proper location of the mathematical process. However, most of these approaches involve charge injection into floating gate devices to adjust the internal impedances or select internal voltages, thus presenting numerous well known difficulties. Some of the difficulties presented are lack of precision of the charge injection, verification of the charge injected, and leakage of the charge injected resulting in drifting of the weights. As a result, there is a need for a neural network that may be easily trained without the need to adjust the network's internal weights or gains.
Neural network training is also difficult because training data sets generally contain a vast quantity of information even though only a small fraction of the information is relevant to carry out the desired task. Processing such a large amount of information requires a great amount of time and requires expending a large amount of computing power. As the task to be carried out by a particular neural network increases, the time and computing power expended to perform the task also increases, eventually reaching the point where any advantages associated with using a neural network to perform the task are lost.
In order to more efficiently process complex information using neural networks, attempts have been made to reuse the results of previous training efforts and computer time so that similar classes of complex problems can be solved without re-teaching a random network from scratch (tabula rasa). Currently, this is done by either training unique networks for each similar task or by training one network “A” to complete one of the tasks and then retraining the resulting network to perform a similar task as network “B”. However, these methods have not proven to be desirable as the retraining of network “A” often takes more time than the training of an empty, random, untrained network. Further, there is no guarantee that A's training is of any use to the training of B as any similarity of tasks may only be in the eye of the user. Finally, the retraining of prior networks may disturb or destroy the arrangement of weights and cause the network to not be useful for its desired task.
Consequently, to make the use of a neural network viable for processing complex operations, a need exists to be able to reduce the amount of training that must be done in order to carry out each operation. More specifically, there exists a need to reuse the results of previous training efforts and computer time so that similar classes of problems can be solved without re-teaching a random network from scratch. Further, there is a need to carryout this training without adjusting the internal weights or gains of the network.